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Abstract 

We define the domain of a linear fractional transformation in a space 
of operators and show that both the affine automorphisms and the compo- 
sitions of symmetries act transitively on these domains. Further, we show 
that Liouville's theorem holds for domains of linear fractional transforma- 
tions and, with an additional trace class condition, so does the Riemann 
removable singularities theorem. We also show that every biholomorphic 
mapping of the operator domain I < Z*Z is a linear isometry when the 
space of operators is a complex Jordan subalgebra of C{H) with the re- 
movable singularity property and that every biholomorphic mapping of the 
operator domain I + Z^Zi < Z2Z2 is a linear map obtained by multiplica- 
tion on the left and right by J-unitary and unitary operators, respectively. 

0. Introduction. This paper introduces a large class of finite and infinite 
dimensional symmetric affinely homogeneous domains which are not holomorphi- 
cally equivalent to any bounded domain. These domains are subsets of spaces 
of operators and include domains as diverse as a closed complex subspace of the 
bounded linear operators from one Hilbert space to another, the identity compo- 
nent of the group of invertible operators in a C*-algebra and the complement of 
a hyperplane in a Hilbert space. Each of our domains may be characterized as 
a component of the domain of definition of some linear fractional transformation 
which maps a neighborhood of a point in the component biholomorphically onto 
an open set in the same space. Thus we refer to our domains as domains of linear 
fractional transformations. We show that at any point of such a domain, there 
exists a biholomorphic linear fractional transformation of the domain onto itself 
which is a symmetry at the point. This linear fractional transformation is a gener- 
alization of the Potapov-Ginzburg transformation. Moreover, for any two points 
Zq and Wo in the domain, there exists an affine automorphism of the domain of 
the form (f){Z) = Wq+ A{Z — Zq) B and is a composition of the above symmetries. 
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In fact, we exhibit many different formulae for linear fractional transformations 
that are automorphisms of domains of linear fractional transformations. 

We show that any bounded holomorphic function on the domain of a linear 
fractional transformation is constant and, more generally, that the Kobayashi 
pseudometric vanishes identically on such domains. We also show that if the 
domain of a linear fractional transformation satisfies a trace class condition, then 
a holomorphic function on the domain which is locally bounded with respect to 
points outside the domain has a holomorphic extension to these points. Further, 
we give a sufficient condition for two domains of linear fractional transformations 
to be affinely equivalent when the subspace of operators considered is the full 
space or a C*-algebra. As expected, our unbounded symmetric homogeneous 
domains fail to have many of the properties of bounded symmetric domains. For 
example, Cartan's uniqueness theorem may fail, symmetries may not be unique 
and automorphisms may not be a composition of a linear fractional transformation 
and a linear map. 

Next we turn to a discussion of the characterization of the automorphisms of 
non-homogeneous domains which are operator analogues of the domain 1 + < 
1^2!^ in (which is included). The domains we consider are circular domains in a 
space of operators in the sense discussed previously by the author in and are 
holomorphically equivalent to the open unit ball of the space with certain singular 
points omitted. We show that with some restrictions, these points are removable 
singularities and we deduce that all biholomorphic mappings of the domains are 
linear. In the case where the domain and range spaces of the operators considered 
have different dimensions, these linear maps are given by operator matrices which 
are the coefficient matrices of linear fractional transformations of the unit ball 
of the space of operators on the domain space. In the case where the operators 
considered have the same domain and range, our domains reduce to analogues 
of the exterior of the unit disc in spaces of operators and the linear maps are 
invertible isometrics of these spaces. A result of Arazy and Solel [0] allows us to 
treat spaces of operators which are not necessarily closed under adjoints. 

A main lemma is the result that every J*-algebra with identity which is closed 
in the weak operator topology has what we call the removable singularity prop- 
erty, i.e., the singular operators in the open unit ball are removable singularities 
for bounded holomorphic functions. This is rather surprising since these singu- 
lar operators may have non-empty interior in the infinite dimensional case. We 
also give two types of homogeneous circular domains where the group of linear 
automorphisms of the domains acts transitively. 

Unbounded homogeneous domains in C"" have been considered previously by 
Penney ||2^ and Winkelmann |^ . In particular, Penney obtains a classification of 
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the rationally homogeneous domains in C" analogous to the classification theorem 
for bounded homogeneous domains given by Vinberg, Gindikin and Piatetskii- 
Shapiro. See also Hua HTSj . 

Our discussion emphasizes the explicit construction of domains and mappings 
using ideas from operator theory and functional analysis. We have considered 
bounded symmetric homogeneous domains from this point of view previously 
in [0. See and for an exposition of the theory of bounded symmetric 



domains in infinite dimensions. 

Let H and K be complex Hilbert spaces and let C{H,K) denote the Banach 
space of all bounded linear transformations from H to K with the operator norm. 
We write C{H) for C{H,H). Throughout, 21 and ^ denote any closed complex 
subspaces of C{H,K). (The reader interested only in the finite dimensional case 
may take H = C", K = C" and identify C{H, K) with the vector space of all 
m X n matrices of complex numbers.) 

1. Domains of Linear Fractional Transformations. Let C E C{K,H), 
D G C{H) and suppose {CZq + D)~^ exists for some Zq G 21. Put Xq = {CZq + 
D)-^C. If ZXqZ G 21 for all Z G 21, we define 

V = Comp^jZ G 21 : {CZ + D)-^ exists} (1) 

to be the domain of a linear fractional transformation on 21. (Here and elsewhere, 
Comp^:^ denotes the connected component containing Zq.) It is easy to show that 
any finite or infinite product of domains of linear fractional transformations is 
the domain of a linear fractional transformation. To justify this terminology, we 
observe that by [0, Prop. 5], if the linear fractional transformation 

T{Z) = {AZ + B) {CZ + D)-^ (2) 

maps a domain in 21 containing Zq onto a subset of some ^ containing an interior 
point of 05 and if the coefficient matrix of T is invertible, then ZXqZ G 21 for all 
Z G 21 and T is a biholomorphic mapping of the domain P above onto a similar 
domain in We consider components because a linear fractional transformation 
may be a biholomorphic mapping of one component of the open set where it is 
defined while mapping the other components outside the space. (See Example 1 
below.) 

Theorem 1 The domain V of a linear fractional transformation on ^ is a sym- 
metric affinely homogeneous domain. In particular, ifT> is given by (j^, then for 
each Y eT>, 

Uy{Z)=Y -{Z-Y){CZ + D)-\CY + D) (3) 
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is a biholomorphic mapping of V onto itself satisfying Uy = I, UyiX) = Y and 
DUy{Y) = —J. Moreover, for any point Wq G V, there is an invertible affine 
linear fractional transformation (pofD onto itself with (plZo) = Wq and (p can be 
chosen to be a composition of mappings Uy given above. 

Winkelmann Prop. 1] has proved a related result for the complement 
of an algebraic subvariety in C". It is easy to show that Uy is the only linear 
fractional transformation (|^) with the mapping property described there which 
satisfies T{Y) = Y and DT{Y) = —I on 21. The transformations Uy are used 
in |]T5| to obtain factorizations of certain operator matrices. 



Example 0. Any 2t is the domain of a linear fractional transformation on 
2t when C = and D = I since then Xq = for any choice of Zq. In this case, 
Uy{Z) = 2Y - Z. 

Example 1. Let 21 be a power algebra i.e., 2t is a closed complex subspace 
of C{H) containing the identity operator I on H and the squares of each of its 
elements. Put 

G/(2l) = Comp^{Z G 21 : exists}. (4) 

Clearly, G/(2l) is the domain of a linear fractional transformation on 21 since 
Xo = / for Zo = I. Also, Uy{Z) = YZ-^Y. If 21 is a Vr*-algebra, the argument 
in 0, Cor. 5.30] shows that the invertible operators in 21 are connected so it is 
not necessary to take components in this case. However, if 21 is the closure of the 
polynomials in the bilateral shift P, Example 4.25], then Uj is a biholomorphic 
mapping of (5/(21) but Uj does not take any other invertible elements of 21 (such 
as the bilateral shift) to 21. 

Example 2. Let 21 be a closed complex subspace of C{H) containing a 
projection E such that ZEZ G 21 whenever Z G 21. Then 

V = Comp^{Z G 21 : {EZ + 1 - E^^ exists} 

is the domain of a linear fractional transformation on 21 since Xq = E for Zq = E. 
The previous examples are the cases E = and E = I. Moreover, 

Ue{Z) = [{E - I)Z + E]{EZ + I- E)-' 

is a variant form of the Potapov-Ginzburg transformation §3]. (Another variant 
is called the Redheffer transform in p. 269]. An early reference is p. 240].) 
A basic property is that Ue is a biholomorphic mapping of VidV onto V2 fl V, 
where 

Vi = {Ze^: Z*JZ<J}, V2 = {Ze%: ||Z||<1}, 
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and J = I — 2E. This follows directly from Lemma 7] since the coefficient 
matrix M of Ue satisfies J2 = (M"^)* JiM-\ where 



Jl 



J 
-J 



J2 



I 
-/ 



Example 3. If ran C is closed or if ran D C ran C, then 

V = {Z e C{H, K) : {CZ + D)-^ exists} 

is the domain of a linear fractional transformation on 2t = C{H, K) when V is 
not empty since 21 is obviously closed under the required products and since V is 
connected by Proposition ^ below. 

Example 4. Let cE H with c 7^ and d G C Then 

V = {z e H : {z,c) ^ -d} 

is the complement of a hyperplane in H and the domain of a linear fractional 
transformation on H with 



Uy{z) 



{y,c) + d 

y--( — r—i\^-y)- 

[z, c) + a 



(Identify H with £(C, H) and apply Example 3 with C = c*.) 

Example 5. Let 21 contain all the rank one operators in C{H, K) and let 
X & H, y E K he unit vectors. Then if ci 7^ 0, 

V = {Z e^: {Zx,y) ^ -d} 

is the domain of a linear fractional transformation on 21. Indeed, let C = xy*, 
D = dl and note that CZ + D = dl + x{y* Z) is invertible if and only if d+{y*Z)x 
is invertible. Also Xq = xy* for Zq = (1 — d)yx* and 



UviZ) = Y 



Z-Y 
d 



xy*Z 



d + (Zx, y) 



{dI + xy*Y). 



z and let c E H 



Since C is compact, T> is connected by Proposition ^ below. 

Example 6. Let if be a Hilbert space with conjugation z 
with c 7^ 0. Then each of the domains 

Vi = {zeH: l + 2{z,c) + {z,z){c,c) ^0}, 
V2 = {zeH: {z,z)^0} 

is linearly equivalent to the domain of a linear fractional transformation. To see 
this, observe that by [0, §2] there exists an invertible linear map z —* of H 
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onto a Cartan factor 2t of type IV. For Vi, take C = A*, Z = Az and D = I, and 
note that CZ + D is invertible if and only if 2 e Pi by [0, (9)]. Also, Xq = A* 
for Zq = so ZXqZ G 21 for all Z G 21 since 21 is a J*-algebra. Moreover, Vi 
is connected by Proposition ^ below since XqZ satisfies a quadratic polynomial 
equation. A similar argument with C = I and D = establishes the result for 
T>2, and in that case. 

Proof of Theorem 1. Given F G P, put X = {CY + D)-^^ and note the 
identities 

CUy{Z) + D = {CY + D){CZ + D)-\CY + D), (5) 
{CY + D)-\CZ + D) = I + X{Z-Y). (6) 



The coefficient matrix of f/y is 
M = 



(I-YX) 2Y-YXY 
X I-XY 



by (D. Clearly = / so M is invertible and if^ = I. Also, Uy{Y) = Y. Hence 
by hypothesis and 0, Prop. 5], f/y is a biholomorphic mapping of V onto itself 
when Y = Zq and therefore ZXZ G 21 whenever Z G 21. Thus Prop. 5] 
applies again to show that Uy is a biholomorphic mapping of T> onto itself for 
arbitrary Y eV. Further, DUyiY) = -I since L'f/y(r)Z = f/y(r + tZ)|t=o 
and UriY + tZ) = Y - tZ{I + tXZ)-^. 

Next we show that ii Z,W eV satisfy ||(CZ + - Z)|| < 1, then 

there is a F G P with Uy{Z) = W. Put X = (CZ + Dy^C and R = W - Z. 
By the holomorphic functional calculus §5.2], the binomial series defines a 
Q G C{H) such that = I + XR, a{Q) does not contain —1, and (/ + Q)~^ is 
a limit of polynomials in XR. Take Y = Z + R{I + Q)-^ . Then F G 21 by 1T|, 
Lemma 6a]. Also, since XR = — I, we have that 

Uy{Z)-Z = Y - Z + {Y - Z)[I + X{Y - Z)] 

= RiI + Q)-'[2iI + Q)+XR]iI + Q)-^ 
= R 

so Uy{Z) = W. Moreover, {CY + D)^^ exists by this and (|^). To show that 
y G P, let < t < 1. The above argument applies to the points Z and Wt = 
(1 — t)Z + tW and the corresponding Qt is a continuous function of t in [0, 1]. 
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Therefore, there is a curve 7 in 21 connecting Z to Y such that Cj{t) + D is 
invertible for all < t < 1. 

Now given Wo G T>, there is a curve 7 in D connecting Zq to Wo- By compact- 
ness, there is a number M such that ||(C7(t) + D)~^C\\ < M for all < t < 1 
and there is a 5 > such that ||7(s) — 7(t)|| < 1/M whenever |s — t| < 6 and 
< s,t < 1. Choose n > 1/6 and put Z/^ = '-y[k/n) for < k < n. Then for 
l<k<n, 

\\iCZk-i + D)-'CiZk-Zk-i)\\<l, (7) 

so by what we have shown there exists a Uk with Uk{Zk-i) = Zk- Thus U = 
Ui---Un satisfies f/(Zo) = Wo- 

To complete the proof, it suffices to show that Uy/ o Uy is an affine linear 
fractional transformation for any W,Y ^T> since then we can choose n to be even 
in the above argument and take (j) = U . Put X = {CY + D)~^C and set 

Q=[W -Uy{Z)][I + X{Z-Y)]. 

Then Q = W -Y + [I + {W - Y)X]{Z - Y) by (1). Hence by (|) and (D, 

UwoUy{Z) = W + Q{CY + D)-\CW + D) 

= Uw{Y) + [I+{W- Y)X]{Z -Y)[I + X{W - Y)], 

as required. □ 
There are many biholomorphic mappings of the domain of a linear fractional 
transformation on 21 besides (^. For example, if Wo G V and | \Xo(Wo — Zo) 1 1 < 1, 
then 

(f){Z) = Wo + [I+ {Wo - Zo)Xo\^l\Z - Zo)\l + Xo{Wo - Zof'' 

is an invertible affine linear fractional transformation of T> onto itself with (^iZo) = 
Wo, where the square roots are defined by the binomial series. This follows 
from [|14| , Lemma 6c], (|^) and the identity 

/ + Xo[(j){Z) - Zo] = R"^[I + Xo{Z - Zo)]R}'\ 

where R = I + Xo{Wo — Zq). Moreover, the above proof shows that finite com- 
positions of these mappings act transitively on T>. Note that (p = Uw ° Uy when 
Wo = Uw{Y) and Zq = F. If 21 = C{H, K) or if 21 is a C*-algebra with C, D g 21, 
a simpler formula for a transitive set of affine mappings of V is given by (12) below 
with R = I. 
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There is a general class of involutory biholomorphic mappings of T> which 
contains the transformations (j^). Specifically, if Wq G V and | |Xo(l^o ~ Zq) | | < 1, 
define 



Vw,{Z) 



Wo-Zo 



{Z-Zo 



where 



Ta{Z) = {I + AXo)-'/\Z -A){I + XoZ)-\I + XoA)'/'. 



(Compare JlT], p. 146].) Note the identities Vwo = o Uzq and Uzq = VVo ° <P- K 
follows that Vwo is a biholomorphic mapping of V onto itself with = I and 
Vwo{Zq) = Wq and that the composition Vwi o Vwo of any two such mappings is 
affine. In particular, Vz^ = Uz^- 

The domain of a linear fractional transformation may have biholomorphic 
mappings which cannot be expressed as the composition of a linear fractional 
transformation and an invertible linear map. For example, let 21 = £(C,C^), 
C = [ 1 ] and D = 0. Then the domain V of d) is 



V = { 



Z2 



h 


Zl 




Zl + g{z2) 




. ^2 . 




Z2 



and 



is a biholomorphic mapping of T) for any holomorphic function g : €,\ {0} C. 
Note that Cartan's uniqueness theorem fails for V since if g{z) = {z — 1)^ and 

^ , then h^Zo) = Zq and Dh^Zo) = I, but h ^ I on V. Moreover, f/^o 

and o Uzq o h are distinct symmetries of V at Zq. 
Let X be any Banach space. 

Theorem 2 IfD is the domain of a linear fractional transformation on 21 and if 
h : V ^ 3C is a bounded holomorphic function, then h is constant. In particular, 
T> is not holomorphically equivalent to a bounded domain. 

Proof. Let Z G 21 with \\Xo{Z - Zq)\\ < 1. We will construct an entire 
function f : € ^ V with /(O) = Zq and /(I) = Z. Then ^(A) = h{f{X)) is 
a bounded entire function so h{Z) = H^Zq) since g is constant Th. 3.13.2]. 
Hence h is constant in a neighborhood of Zq so h is constant on V by the identity 
theorem [0, Th. 3.16.4]. 
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Recall M, §6.3] that limsup^_ 



l/n 



< 1 and that the binomial 



series bx{z) = J2'^=o ynj satisfies bx{z)b_x{z) = 1 for all A G C and 

1 2; I < 1. Put W = Xq{Z — Zq). By the holomorphic functional calculus, 
bxiW)b-xiW) = I and hence bxlw)-^ exists for all A G C. Define /(A) = 

Zo+{Z- Zo) E^=i (^n) ^^^^ f ^^^^^^ ■^(^) ^ ^ all A G C 

by P^ , Lemma 6a]. Moreover, if A G C, then 

(CZo + D)-\Cf{\) + D]=I + Xo[/(A) - Zo] = bx{W), 

so /(A) G I). □ 
Note that it follows from the properties of /(A) given above and (|^) that the 



pseudometric p assigned to V by any Schwarz-Pick System |jT2[ satisfies p = 0. 
Thus, in particular, the Caratheodory and Kobayashi pseudometrics for T> vanish 
identically. (Compare Cor. 1].) 

The next result gives a class of domains where all points outside the domain 
are removable singularities for locally bounded holomorphic functions. 

Theorem 3 Suppose the set V = {Z e ^ : [CZ + D)'^ exists} is non-empty 
and C is a trace class operator. Then is an analytic set. Moreover, if S is 

a domain in 2t and if h : V (1 S ^ 3C is a holomorphic function which is locally 
bounded in the sense that each operator in S \ V has a neighborhood U such that 
h is hounded onU (IV, then h extends to a holomorphic function h : S ^ 3C-. 

It is easy to show that the conclusions of Theorem |^ hold for any T> satisfying 
the hypotheses of Theorem ^ See ||2^ for the definition and properties of analytic 



sets in Banach spaces. Without some restriction on C or 21, the complement of 
V may have non-empty interior and thus is not an analytic set. For example, if 
there is a Zi G 21 such that CZi + D has only a one-sided inverse Q, then an 
elementary argument shows that CZ + D is not invertible for any Z G 21 with 
\\Z-Zi\\< 1/r, where r = ||C|| ||Q||. 

Proof. By (j^) and a translation, we may suppose that D = I. It suffices to 
show that there is a holomorphic function / : 21 ^ C satisfying 

I? = {Z G 21 : fiZ) ^ 0} (8) 

smce then 21 \ P is an analytic set and the remainder of the theorem follows from 



the extension of the Riemann removable singularities theorem given in |23, Th. 
II. 1.1. 5]. It is easy to construct such an / when C has finite rank. Indeed, put 
Hi = ran C, Ki = (kerC)-*", H2 = and K2 = K^. Since these spaces are 
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closed, the operators in C{H, K) and C{K, H) can be written as associated 2x2 
operator matrices. In particular, if Z G C{H,K), 





Z2 ' 


, c = 


' Y 


' 


Z3 


z. 









(9) 



where Z^ E C{Hr,K{), E C{H2,K^), ^3 e CiH^.K^), Z^ E CiH^, K2) and 
Y E C{Ki,Hi). Hence 



I + CZ 



I + YZi YZ2 
/ 



(10) 



where the successive appearances of / denote the identity operator on H, Hi and 
H2, respectively. Thus Z E V if and only if (/ + YZi)~^ exists. Since Hi is finite 
dimensional, we may take f{Z) = det(J + YZi) to obtain 

If C is a trace class operator, there exists a sequence {Fn} of finite rank 
operators in C{H, K) with | |F„ — C| |i — > 0, where 1 1 | |i is the trace class norm H, 
Th. VI. 4.1]. Let /„ be the function defined above where C is replaced by F„. 
Then by H, p. 115-117], the sequence {/„} is locally uniformly bounded on 2t and 
converges pointwise to a function / on 2t which is denoted by f{Z) = det(J+CZ). 
Hence / is holomorphic on 21 by an extension of Vitali's theorem |1T7| , Th. 3.18.1] 
and d) holds by [|, VII.3(13)]. □ 

In some cases it is not necessary to take components in the definition of the 
domain of a linear fractional transformation. 



Proposition 4 The set 

V = {Ze%: + exists} (11) 

is connected if at least one of the following conditions holds: 

a) C2t contains only compact operators, 

b) there is some Zq eT> such that each operator in XqSI satisfies a non-trivial 
polynomial equation, 

21 = C.{H, K) and ran C is closed, 

d) ^ = C{H, K) and ran D C ran C. 
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Here ran denotes the range of operators. It is not difficult to deduce from 



and [0, Lemma 18] that the domain T) above is a circular domain in the sense 
of when (a) or (b) holds. (Take J = - [ C D ]* [ C D ].) 

Proof. Suppose that (a) holds and that V is not empty. As in the proof of 
Theorem ^ we may suppose that D = I. Let Z E V and put f2 = {/x G C : e 
V}. Clearly fl contains and 1, and the set (D \f2 is discrete by hypothesis. Hence 
there is a curve uj in Q connecting and 1 so 7 = ujZ is a curve in V connecting 
and Z. Therefore, V is connected. A similar argument shows that T> is connected 
when (b) holds. 

Suppose that (c) holds. As in the proof of Theorem ^ we may suppose that 
D = I. Moreover, the decompositions (§) hold, where Y G C{Ki, Hi) is invertible 
and Hi is a closed subspace of H. Let Z eV. Then (J + YZi)^^ exists so by the 
connectedness [|^, Cor. 5.30] of the set G of invertible elements of C{Hi), there 
exists a curve u in G which connects / to / + YZi. Define 



lit) 



Y-^u;{t)-Y-^ tZ2 
tZ^ tZ/i 



< t < 1. 



Then / + C'-f{t) is invertible for < t < 1 by (|T0|) so 7(t) is a curve in I) which 
connects to Z. Thus T> is connected. 

Suppose that (d) holds and that Zq G V. By [|], there is a Zi G C{H, K) with 
C Zi = D so C{Zi + Zq) is invertible. Then ran C = H and hence T> is connected 
by part (c). □ 

The following gives a condition under which two domains of the form (pAl) are 
affinely equivalent. 

Proposition 5 Let Ci, C2 G C{K, H), Di,D2e C{H) and put 

Vi = {ZeC{H,K): (CiZ + Di)-^ exists}, 
V2 = {Z e C{H, K) : {C2Z + D2Y^ exists}. 

Suppose there exists an invertible R G C{K) with C2 = CiR. Then for each 
Zi G Vi and Z2 G V2 there is an invertible affine linear fractional transformation 
(f) ofT>i onto V2 with (f){Zi) = Z2- 

Proof. Observe that 

(/.(Z) = + R-\Z - Zi){CiZi + Di)-\C2Z2 + D2) (12) 
is the required affine mapping since 

C2<\>{Z) + D2 = {CiZ + Di){CiZi + Di)-\C2Z2 + D2). □ 
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Note that Proposition |^ and its proof also hold when all the spaces of operators 
mentioned are replaced by the same C*-algebra. 

2. Linear Automorphisms of Domains. The main results of this section 
are that certain unbounded circular domains (in the sense of fl^) have only 
linear automorphisms. In the finite dimensional case, the domains we consider 
are incomplete matrix Reinhardt domains and extensions of these to non-square 
matrices. (See ^4].) Throughout, we let 2to denote the open unit ball of 21 in 
the operator norm. Thus 

21o = {Zg21: \\Z\\<1}. 



Theorem 6 Suppose H is finite dimensional and K ^ {0}. Let 21 = C{H, K x H) 
and put 

V = {z e%: I + ziZi < z;z2}, 



where Z 



Zi 

Z2 



is the decomposition of Z with Z\ G C{H,K) and Z2 G jC{H). 



Then h is a biholomorphic map ofD onto itself if and only h{Z) = LZU , where 
L is an invertible linear map in C{K x H) satisfying L*JL = J, J ' ^ ^ 



and U G C{H) is unitary. 







Corollary 7 Suppose K 7^ {0}. Then h 
morphic mapping of 



z 




' A b' 




z 


A 




c* d 




A 



is a biholo- 



V = { 



z 
A 



if and only if T{z 



eKx C: l + \\z 
Az + b 



is a biholomorphic mapping of Kq = {z G 



K : \\z\\ < 1}, where the coefficients of T have been multiplied by a positive 
number so that \d\'^ — = 1. Moreover, there are no other biholomorphic 
mappings of V or Kq. 

Corollary |^ follows from Theorem ^ the form of biholomorphic mappings 
of Kq, and the uniqueness (up to a complex scalar) of the coefficients of linear 
fractional transformations. (See ([131) below and extend |^1], Th. 3].) Note that 
Theorem || is no longer true when K = {0} (i.e., Zi does not appear) since 
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h{Z) = is a biholomorphic mapping of V which is not of the specified form. 
We consider the case K = {0} next. 

Let 21 be a power algebra. We say that 21 has the removable singularity 
property if every bounded holomorphic function h : (^/(Sl) fl 2to — >■ X extends to 
a holomorphic function on 2lo for any Banach space X. It follows from ^ that 
there is no loss of generality if one takes X = C. Thus the classical Riemann 
removable singularities theorem for several complex variables shows that 21 has 
the removable singularity property whenever dim 21 < oo. Recall ||T^ that a J*- 
algebra with identity may be characterized as a power algebra which contains the 
adjoints of each of its elements. 

Proposition 8 Every J*-algebra with identity which is closed in the weak opera- 
tor topology has the removable singularity property. 



Theorem 9 Let 2t be a power algebra with the removable singularity property 
and put 

V = {Z e Gi{^) : I < Z*Z}. 

Then h is a biholomorphic mapping of V onto itself if and only if h = L, where 
L is a linear isometry ofOi onto itself with L{I) G G/(2t). 

Thus by Proposition ^ the conclusions of Theorem ^ hold, for example, for 
C{H), any iy*-algebra, any finite rank J*-algebra having a unitary element, any 
Cartan factor of type II or IV and any Cartan factor of type III where the dimen- 
sion of the underlying Hilbert space is even or infinite. Note that the condition 
I < Z*Z in the definition of V above can also be written as / < ZZ* since for 
Z e G/(2l), the identity ||(^-^)*|| = \\Z-^\\ holds and / < Z*Z if and only if 
< 1. The domains V of Theorems ^ and ^ are not homogeneous. To see 



rl 

By Theorem ^ if /i is a biholomorphic mapping of V with h{Z^) = Zg, then 
/ + Z*JZs = U*{I + Z;jZr)U so s = r. The case of Theorem | is similar. 

Proof of Theorem Suppose h has the given form h{Z) = LZU. Then 
h{Z)*Jh{Z) = U*Z*JZU and 

V = {Ze%: I + Z*JZ<0}, 

so h{V) C V. Since h^^{Z) = L^^ZU^^ has the same form as h, it follows that 
/i is a biholomorphic mapping of T>. 



this when T> is as in Theorem 0, observe that Zr 



is in T> for r > 1. 
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Now suppose his a. biholomorphic mapping of V and put £ = {Z E ^ : exists}. 
Clearly V <Z £ since dim H < oo and note that Z G 2lo if and only if Z G 2t and 



Z*Z < I. Hence T 
%n£ with = 



^1 

^2 



Z1Z2 
^2 



is a biholomorphic mapping of T> onto 



T. Therefore, g = TohoTisa biholomorphic mapping of 
2lo n £. By Theorem ^ with C = [ / ], (7 extends to a holomorphic mapping 
: 2lo ^ 21. To show that ^(2lo) C 2lo, suppose this is false. Then by the 
maximum principle Th. 3.18.4], there is a Zq G 2to with 11^(^0)11 > 1- Let 
u,v E H he unit vectors with {g{Zo)u, f ) > 1 and define 



i[giZ)-giZo)]u,v) 



Then / is holomorphic on 2lo fl £ but / does not have a holomorphic extension 
to 2lo, contradicting Theorem |^. Similarly, g~^ extends to a holomorphic map 
of 2lo into 2lo which is g~^ by the identity theorem [|1^, Th. 3.16.4]. Hence g is 
a biholomorphic mapping of 2lo which takes £ onto itself. (Compare ||27| , 
Lemma 2.4.4].) 

By d^, Th. 5.3], we may write 



g{Z) = {AZ + B){CZ + Dy 



M 



A 
C 



B 
D 



where M is an invertible operator in C{{K x H) x H) satisfying M*JiM 
Irxh 



(13) 



Ji 



and Ji 







D G C{H). Let 



. Here A G /:(ir x //), 5 G 2t, C G CiK x ff) and 



" A, 


A2 ' 


, B = 


' 5i " 


As 


A, 


^2 



c 



Ci C2 



be the corresponding decompositions. It suffices to show that A2 = 0, A3 = 0, 
B2 = 0, C2 = and ^ exists; for then the identity h = T o g o T gives h{Z) = 

Ai B, 



LZU , where L 



and U = Al\ and it follows from M*JiM = Ji 



Ci D 

that L and U asserted. 

Pick an r with < r < 1 and fix Zi G C{H, K) with ||^i|| < yl — r^. Define 
(l){Z2) = A3Z1 + A4Z2 + B2 for Z2 G C{H). By the mapping properties of g, if 
1 1^2 1 1 < r, then (j){Z2) is invertible whenever Z2 is invertible and (t){Z2) is singular 
whenever Z2 is singular. Since the determinant is analytic, it follows from the 
identity theorem that 0(^2) is singular for all singular Z2. 
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If A4 is not invertible, there is a unit vector u E H with A^u = 0. Let Z2 
be invertible with \\Z2\\ < r and let Z2 = {I — uu*)Z2. Then Z2 is singular 
since u*Z2 = 0, and 0(^2) = ^(^2), a contradiction. Thus A4 is invertible. Put 
B = Al^{A^Zi + B2). Then Z2 + B is singular whenever Z2 is singular. To show 
that i? = 0, let M G -ff be a unit vector and take Z2 = (/ — — un*). Then 
Z2 is singular since Z2U = 0soZ2 + B = I + {Bu — u)u* is singular. Hence, 
{Bu,u) = l + u*{Bu -u) = 0. 

We have shown that A^^ exists and that A3Z1 + B2 = for \ \Zi 



A3 = and B2 = 0. Now the coefficient matrix of ^ ^ is M ^ 



< vT^^r^, so 

A* -C* 
-B* D* 



and hence the above argument applied to g^^ shows that A2 = and C2 = 0, as 
required. □ 
We deduce Theorem ^ from the following Lemma. 

Lemma 10 Let 'Qi be a power algebra and let L be a linear isometry of 21 onto 
itself. Then U = L{I) is unitary, U* G 21, L(G'/(2l)) is a component of the set 
of invertible operators in 21 and 

L{Z-^) = UL{Z)-^U (14) 
for allZ e GjCQI). 

Proof. By 0, Cor. 2.8], f/* G 21 so by |, Th. 2], U is unitary and 
L[Z) = Up{Z), where p : 21 ^ f/*2l is an invertible linear isometry satisfying 
p{I) = I and p(Z2) = p{Zf for all Z G 21. Hence 

piZWZ) = p{Z)p{W)p{Z) (15) 

for all Z, G 2t since 

ZWZ = 2{Z oW)o Z - {Z o Z)oW, (16) 

where Z o W = [{Z + W)"^ - Z^ - W'^]/2. Let Z G ^7(21). Then Z'^ G Gi{^) 
by Example 1. Hence it follows from (|T5|) and the identities / = Z{Z^^)^Z and 
Z = ZZ~^Z that p{Z)~^ exists and p{Z~^) = p{Z)~^. Thus L{Z)~^ exists and 
(lU) holds. Also, L(G/(2l)) is a component of (7(21) since L is a homeomorphism. 



□ 



Proof of Theorem |9|. Let L : 21 — > 21 be an invertible linear isometry with 
L{I) G G7(2t). Then by Lemma |T0|, L(G/(2t)) = Gi{^) so L is a biholomorphic 
mapping of £^ = 2to fl G'/(2t). By Example 1, T{Z) = Z~^ is a biholomorphic 
mapping of T> onto S and hence g = ToLoTisa, biholomorphic mapping of 
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V. Moreover, L{Z) = Ug{Z)U for all Z G G/(2t) by (0). Therefore, L is a 
biholomorphic mapping of V. 

Now let h he a. biholomorphic mapping of T>. Then g = TohoTisa. biholo- 
morphic mapping of S. By hypothesis and the argument given in the proof of 
Theorem ^, g extends to a biholomorphic mapping g of SIq. Put B = —g~^{0). 
Clearly B G SIq and it follows from Theorem 2.2(ii) and Theorem 2.6 of that 
B* G 21. Hence 

Tb{Z) = (/ - BB*)-^Z + B){I + B*Z)-\I - B*B)^ (17) 
is a biholomorphic mapping of 2to by ([16|) , the comments after Proposition 5 of 



and (12)]. Thus the proof of Theorem 3 of applies to show that 
g = L oTb, where L : 21 — 21 is an invertible linear isometry. By Lemma |l^, 
takes G/(2l) into G(2l) so Z + S is invertible whenever Z G Gj{^) and \ \Z\ \ < 1. 
We will show that B = 0. Then g = L and /i = To(yioTsoby Lemma IC, h = Li 
on P, where Li{Z) = VL{Z)V and V = L{I)* is unitary. Hence Li is a linear 
isometry on 21 and Li(2l) = 2t since Li = Dh{21). Moreover, Li(/) G G'7-(2l) 
since h{21) G ^7(21). 

To show that B = 0, let U he any unitary operator in G/(2t) and note that 
U* G ^7(21) since U* = T{U). By what we have shown above, {-XU* + B)-^ 
exists for all |A| < 1 so \UB\c, = 0, where | \a denotes the spectral radius. The 
straightforward extension of [|10], Prop. 2a] to 21 shows that Tb* (A/) is a unitary 
operator in (7/(21) for all |A| = 1. Thus /(A) = Tb*{\I)B is an operator-valued 
function which is holomorphic on a neighborhood of the closed unit disc with 



|/(A)|o- = for all |A| = 1 so |/(0)|o- = by a maximum principle of Vesentini EE 



Therefore, \B*B\(j = so -B = 0, as required. □ 
Proof of Proposition |8|. Put £ = 2lo fl ^7(21) and let h : £ ^ 'X he a 
holomorphic function with ||/;,(Z)|| < 1 for all Z E S. Given < t < 1/2, put 
r = l-t. Let G 2lo and suppose a{W) is a finite set. Then /(A) = h{tI + XW) 
is a vector- valued function with unit bound which is holomorphic at all but a finite 
number of points in the disc |A| < r so / has a holomorphic extension to this disc 
by [T^, Th. 3.13.3]. By the Cauchy estimates [0, (3.11.3)], |/(")(0)| < nlr"" so 

||Z)"/i(t/)(W^)|| < n!r~", (18) 

where D'^h{Z) denotes the homogeneous polynomial associated with the nth order 
Frechet derivative at Z. 

By the spectral theorem, each normal operator in 21 is a limit in the operator 
norm of a linear combination W of orthogonal projections in 21 which sum to /. 
Hence (|18D holds for all normal operators W G 2lo. By the maximum principle 
for unitary operators |10|, Th. 9], inequahty ( [T8| ) holds for all W G 2lo. 
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Define 



OO 1 

ht{z) = Y.-wh{ti){z-ti) 



n.=0 



and put T'f = {ZG2t: ||Z — t/||<r}. Then ht is liolomorpliic in Vt since 
by (|TBp the above series converges uniformly on each ball about tl with radius less 
than r. Also, £ is connected and ht{Z) = h{Z) for all Z with ||Z — < t 
by Taylor's theorem Th. 3.17.1]. Since {Dj : < t < 1/2} is a monotone 
family of domains with union 2lo, it follows from the identity theorem |1^, Th. 
3.16.4] that there exists a holomorphic extension h of h to SIq. □ 
In the remainder of this section, we give some types of circular domains which 
are homogeneous under the group of linear automorphisms. Our first result re- 
duces to ||T^, Example 4] when H = C 

Proposition 11 Let 21 = C{H, K x H) and put 

V = {Z ZlZi < ZIZ2, Z^^ exists}. 

Then V is symmetric and the group of linear biholomorphic mappings of V acts 
transitively on V. Moreover, D is holomorphically equivalent to the product do- 
main V' = C{H, K)^ X GC{H), where GC{H) is the set of all invertible operators 
m C{H). 



Proof. Note that V 
I 



and J 







= {Z eS : Z*JZ < 0}, where S = {Z E ^ : Z^^ exists} 
Given W e V, put B = WiW2^ . Since ||i?|| < 1, we may 



define L{Z) = MZR, where 
M = 



{I -BB*)~2 
{I -B*B)-^B* 



B{I - B*B) 
(I- B*B)- 



and R 




(/ - B*By/'^W2. (Here M is the coefficient matrix of (P).) Clearly 
= W. Moreover, M*JM = J so L(V) C V since L{Z)*JL{Z) = 



R*Z*JZR and (/ - B*B)-^/^{B*ZiZ2^ + I)Z2R is invertible for all Z e V. 
Since is obtained from M by replacement of B by —B in the expression for 
M, a similar argument shows that L'^{T>) C V. Therefore, L is a biholomorphic 
mapping of T>. 

It is easy to verify that h(Z) = (ZiZ2^, Z2^) is a biholomorphic mapping of 
T> onto T>' and hence T> is symmetric since each of the factors of T>' is symmetric. 
□ 
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Proposition 12 Let J e with J* — J and suppose 1 and —1 are eigen- 

values for J. Then the group G of linear biholomorphic mappings of V — {z & 
H : [Jz, z) < 0} acts transitively on V. 



Proof. By hypothesis, H — Ce0ir0C/, where e and / are unit eigen- 
vectors for 1 and —1, respectively, and K — {e, /}-*-. Thus if 2; e if, we may 
write 





a 




' 1 








z — 


w 


, J- 





B 

















-1 



V^{zeH: \a\'^+{Bw,w) <\P\'^}. 



Suppose zi E V and let cti, Wi and (3i be the coordinates of zi. We will obtain 
an L e G with Lf = z\. 

First suppose ^ = |q;i| + |/3i| > and put 

a= |q;i| + ^(-Bwi,^!), 6= - ^(5^1,^1). 

Clearly a + 6 = A and A{f) - a) > so 6^ - > 0. Given w e K,let Ly, : H ^ H 
be the linear map defined by the operator matrix 

l-l{Bw,w) -w*B -l{Bw,w) 
w I w 

l{Bw,w) w*B l + \{Bw,w) 

Then L^^ = and L^^JL^ — J so L^j E G. Take w = Wi/A. Then by a short 
computation. 





a 




\ai 









Wi 




_ b _ 







Note that Li 







and Li is in G since 



\/W^^aF satisfies Li/ 
a 6 

U[JLi = (6^ — a^) J and L^^ exists. Also, there is an L2 G G (given by a diagonal 



a 

b 





\ai\ 






matrix) such that L2 






Wi 








. Pi . 



. Hence L2LyjLif — Zi. 
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Now suppose ai — Pi — 0. Then Ly^^zi 



{Bwi,wi) 



and {Bwi, wi) ^ 



since Zi E V. By what we have shown, there is an L e G with Lf — Ly,^zi so 
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